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1.  Introduction 

Simulation-based  training  is  increasingly  important  in 
Navy  training.  However,  replicating  real-world 
environments  has  inherent  challenges  such  as  the 
necessity  to  provide  realistic  human  behaviors  in  the 
simulated  environment.  One  solution  is  to  use  human 
role-players  for  friendly  and  enemy  forces.  However, 
using  role-players  is  costly  in  terms  of  money  used  to  hire 
outside  contractors,  operational  time  foregone  by 
volunteer  role-players,  and  the  added  equipment  for  role- 
players.  Semi-Automated  Forces  (SAFs)  provide  a  less 
costly  alternative  to  replicating  friendly,  enemy,  and 
neutral  platforms  in  the  virtual  environment.  They  are 
controlled  and  monitored  by  a  human  that  pre-scripts 
command  processes  (Department  of  Defense,  1998). 
Although  SAFs  decrease  the  costs  associated  with  using 
human-role  players,  the  pre-scripted  nature  of  their 
behaviors  presents  some  inherent  challenges.  This  paper 
provides  an  overview  of  the  current  state-of-the-art  in 
human  behavior  modeling  and  outlines  remaining 
challenges.  The  authors  then  provide  a  practical 
framework  for  evaluating  rapid  human  behavioral 
modeling  toolsets  to  overcome  the  presented  challenges. 

2.  Challenges  of  Pre-scripted  Behaviors 

While  SAF  behavior  significantly  contributes  to  the 
realism  of  training  scenarios,  limited  behaviors  provide  an 
unrealistic  situation  that  may  hinder  training  transfer 
(Gelenbe,  Hussain,  &  Kaptan,  2005).  This  lack  of  realism 
is  often  because  SAFs  must  be  scripted  prior  to  the 
training  event.  For  this  reason,  many  mission  variations 
are  preprogrammed  to  facilitate  realistic  tactical 
behaviors.  Further,  some  training  scenarios  require 
thousands  of  SAF  entities  that  must  be  pre-scripted  to 
successfully  execute  training.  However,  pre-scripting  this 
many  entities  with  several  behavioral  variations  is 


impractical  due  to  time  constraints  and  increased 
manpower  requirements  (Cox  &  Fu,  2005). 

Even  when  SAF  entities  are  scripted  with  few  behavior 
variations,  scripting  large  numbers  of  SAFs  in  short 
periods  of  time  also  presents  challenges.  There  is  often 
an  increase  in  manpower  to  support  scenario  generation, 
(albeit,  less  than  using  live  role-players)  and  instructors 
work  long  hours  to  ensure  that  training  events  are  kept  on 
schedule.  Increased  work  hours  contribute  to  cognitive 
fatigue  and  thus  could  limit  the  quality  of  training 
provided  by  an  instructor  (Whelan,  Loftus,  Perme,  & 
Baldwin,  2002).  Finally,  as  large  scale  simulation-based 
training  events  become  more  common  and  increase  in 
scale,  additional  instructors  are  required  to  monitor  SAF 
behaviors,  causing  training  costs  to  increase  (Furness  & 
Tyler,  2001). 

3.  Behavior  Modeling  Evaluation 

The  previously  mentioned  challenges  to  SAFs  limit 
fidelity  and  increase  costs,  showing  a  need  to  practically 
evaluate  current  human  behavior  modeling  toolsets  in  a 
manner  that  can  help  overcome  these  challenges.  A 
review  of  current  behavior  modeling  technologies 
indicates  two  prominent  technical  approaches  for  creating 
more  realistic  SAFs:  algorithms  and  hierarchies.  While 
algorithmic  approaches  use  behavioral  instances  to 
capture  demonstrated  behaviors,  hierarchal  approaches 
decompose  high  level  tasks  or  goals  into  primitives  to 
elicit  behaviors.  Both  approaches  of  behavior  modeling 
have  shown  to  be  effective  methods  of  producing  more 
realistic  behaviors  (Banks  &  Stytz,  2003).  While  these 
approaches  are  effective  means  of  modeling  realistic 
behavior,  toolsets  using  these  approaches  should  be 
evaluated  on  several  criteria  to  practically  increase  Return 
on  Investment  and  drive  future  scientific  inquiry. 
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We  have  developed  a  behavior  modeling  toolset 
evaluation  framework  which  can  be  divided  into  three 
categories:  cost,  schedule,  and  performance.  Each 
category  has  its  own  set  of  evaluation  criteria. 

3.1  Cost 

The  cost  category  is  broken  into  three  criteria  thought  to 
reduce  the  cost  of  implementing  a  behavior  modeling 
toolset.  The  three  evaluation  criteria  are: 

1)  Domain  Independence.  Can  entities  be  reused  in  a 
variety  of  training  scenarios  and  simulations  regardless  of 
developmental  domain? 

2)  Technology’  Readiness  Level  (TRL).  What  is  the  level 
of  maturity  of  the  technology? 

3)  Resource  Requirements.  How  much  binding  is 
required  to  increase  product  maturity? 

3.2  Schedule 

The  time  category  consists  of  one  criterion: 

1)  Rapid  Scripting  Capabilities.  Can  the  toolset  rapidly 
script  entity  behaviors? 

3.3  Performance 

The  performance  category  is  focused  on  the  actual 
performance  of  the  entity  or  toolset,  and  consists  of  two 
components: 

1)  Autonomy.  Does  the  toolset  reduce  the  manpower 
required  to  monitor  entities? 

2)  Communication  Capability.  Does  the  toolset  support 
more  realistic  interaction  with  entities? 

4.  Benefits 

There  are  numerous  anticipated  benefits  of  evaluating 
toolsets  using  this  framework.  First,  training  fidelity  and 
transfer  are  expected  to  increase,  as  rapid  scripting 
reduces  the  time  necessary  to  produce  more  behavior 
variations  than  current  SAFs  provide.  Communication 
capabilities  can  also  enhance  realism  by  allowing  the 
trainee  to  simulate  communication  with  entities  (Furness 
&  Tyler,  2001).  Next,  manpower  requirements  are 
expected  to  decrease  as  the  reuse  of  behavior  models  in 
various  training  scenarios  and  simulations  reduces 
scenario  generation  time.  The  production  of  autonomous 
entities  is  expected  to  further  reduce  manpower  costs  by 
reducing  monitoring  requirements.  Costs  are  further 
reduced  by  selecting  toolsets  that  have  higher  TRLs  and 
fewer  resource  requirements.  Finally,  reduction  in 
scenario  generation  time  and  monitoring  requirements  can 
also  alleviate  the  cognitive  strain  placed  on  instructors 
allowing  them  to  focus  on  other  aspects  of  the  training 
scenario,  such  as  performance  measurement. 


Authors’  Note.  The  views  expressed  herein  are  those  of 
the  authors  and  do  not  necessarily  reflect  the  official 
position  of  the  organizations  with  which  they  are 
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